System and method for rendering in accordance with location of virtual objects in real-time

ABSTRACT

There is provided a system and method for rendering in accordance with location of virtual objects in real-time. There is provided a method for persistent association of a graphic overlay with a virtual object in a displayable environment, comprising receiving a first three-dimensional coordinate of the virtual object in the displayable environment, determining a three-dimensional coordinate of the graphic overlay in accordance with the first three-dimensional coordinate of the virtual object, tracking a movement of the virtual object in the displayable environment by receiving one or more second three-dimensional coordinates of the virtual object, and modifying the three-dimensional coordinate of the graphic overlay in accordance with the one or more second three-dimensional coordinates of the virtual object.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates generally to digital video. Moreparticularly, the present invention relates to rendering digital videorendering.

2. Background Art

Sports are widely watched and enjoyed by many people, from dedicatedsports fans to casual spectators. While just watching the game or matchby itself is already exciting, the addition of supplemental commentary,statistics, analysis, and visuals may deepen viewer appreciation of thetechniques and strategies used by the sports players. Viewers can keeptabs on the running statistics of their favorite players, viewsimulations of various potential game plans, and learn about theplanning and strategy involved in past replays. In this manner, newspectators can quickly get up to speed on game rules and commonscenarios, whereas seasoned fans can deepen their knowledge of specificstrategies and detailed statistics.

Some examples of such supplemental content may include statisticsmarquees, player profile screens, and other information. However, muchof this supplemental content is traditionally presented either on areserved static portion of the screen or on a separate screen. Separatescreens may distract viewers from primary footage of the game, whereasstatic locations might be difficult for viewers to read and focus. Bothsituations are less than ideal. On the other hand, placing suchsupplemental content on a more flexible, dynamically locatable overlaytraditionally requires manual path adjustment to avoid interfering andobscuring other onscreen objects, a potentially time consuming and errorprone process.

Accordingly, there is a need to overcome the drawbacks and deficienciesin the art by providing a way to present dynamically moving overlayswithout requiring manual positioning adjustments.

SUMMARY OF THE INVENTION

There are provided systems and methods for rendering in accordance withlocation of virtual objects in real-time, substantially as shown inand/or described in connection with at least one of the figures, as setforth more completely in the claims.

BRIEF DESCRIPTION OF THE DRAWINGS

The features and advantages of the present invention will become morereadily apparent to those ordinarily skilled in the art after reviewingthe following detailed description and accompanying drawings, wherein:

FIG. 1 presents a system for rendering in accordance with location ofvirtual objects in real-time, according to one embodiment of the presentinvention;

FIG. 2 presents a diagram of real and virtual environments for use by asystem for rendering in accordance with location of virtual objects inreal-time, according to one embodiment of the present invention;

FIG. 3 presents a series of exemplary interfaces on a display using asystem for rendering in accordance with location of virtual objects inreal-time, according to one embodiment of the present invention; and

FIG. 4 shows a flowchart describing the steps, according to oneembodiment of the present invention, by which a rendering device canrender in accordance with location of virtual objects in real-time.

DETAILED DESCRIPTION OF THE INVENTION

The present application is directed to a system and method for renderingin accordance with location of virtual objects in real-time. Thefollowing description contains specific information pertaining to theimplementation of the present invention. One skilled in the art willrecognize that the present invention may be implemented in a mannerdifferent from that specifically discussed in the present application.Moreover, some of the specific details of the invention are notdiscussed in order not to obscure the invention. The specific detailsnot described in the present application are within the knowledge of aperson of ordinary skill in the art. The drawings in the presentapplication and their accompanying detailed description are directed tomerely exemplary embodiments of the invention. To maintain brevity,other embodiments of the invention, which use the principles of thepresent invention, are not specifically described in the presentapplication and are not specifically illustrated by the presentdrawings.

FIG. 1 presents a system for rendering in accordance with location ofvirtual objects in real-time, according to one embodiment of the presentinvention. Environment 100 of FIG. 1 includes virtual environment 100,remote virtual simulator 115, virtual environment data 120, realenvironment 130, local video capture system 135, real environment data140, network 150, live broadcast link 155, virtual rendering device 160,composite render 165, and display 170. Virtual environment data 120includes virtual object 121 and virtual render 129. Virtual object 121includes virtual coordinates 122 and metadata 123. Metadata 123 includestranslation data 124, motion data 125, and priority data 126. Realenvironment data 140 includes real object 141 and video capture 149.Real object 141 includes real coordinates 142 and metadata 143. Metadata143 includes translation data 144, motion data 145, and priority data146. Virtual rendering device 160 includes processor 161, graphicoverlay 162, and compositing logic 164. Graphic overlay 162 includesoverlay coordinates 163.

Virtual environment 110 may represent a virtual three-dimensional worldand may include various data such as world settings, three-dimensionalmodels and objects, textures, behaviors, properties, and other data.Remote virtual simulator 115 may then process virtual environment 110 toprovide a virtual simulation. Remote virtual simulator 115 may comprise,for example, a personal computer using standard off-the-shelfcomponents, or a videogame console. Remote virtual simulator 115 maythen execute a customized simulation program for simulating virtualenvironment 110 according to solicited input parameters. For example,the input parameters may be provided by a keyboard, mouse, or game-padconnected directly to remote virtual simulator 115, or relayed overnetwork 150.

While remote virtual simulator 115 is simulating virtual environment110, virtual rendering device 160 may request a one-time or continuouslyupdated data feed from remote virtual simulator 115 through network 15regarding specific or comprehensive aspects of the simulation. As shownin FIG. 1, this data feed is captured as virtual environment data 120,including virtual object 121 and virtual render 129. Although FIG. 1presents remote virtual simulator 115 as a remotely accessible resourcefrom network 150, alternative embodiments may have utilize a locallyaccessible virtual simulator connected to virtual rendering device 160without using network 150, or the functions of remote virtual simulator115 might be incorporated as part of virtual rendering device 160itself.

Virtual object 121 may contain data describing a virtual object withinvirtual environment 110, including the position of the virtual objectwithin virtual three-dimensional space, or virtual coordinates 122, andother data contained in metadata 123. Metadata 123 includes translationdata 124, which may include any scaling, rotation, and orientation datarelevant for the virtual object, motion data 125, which may include, forexample, directional vectors in three-dimensional space or projectedtrajectories, and priority data 126, which may resolve whether thevirtual object should be visible depending on the priorities of otherobjects.

Although only a single virtual object 121 is present within virtualenvironment data 120, alternative embodiments may include severalvirtual objects, or even all the virtual objects from virtualenvironment 110. Additionally, the elements presented within virtualobject 121 are merely exemplary, and other embodiments may includeadditional or alternative data sets. For example, metadata 123 mayinclude additional components, such as statistics, history, previoussimulation records, and pointers to external data.

Virtual render 129 may comprise, for example, a rendered visualrepresentation of the simulation, suitable for outputting to a displaysuch as display 170. Although virtual render 129 is shown as provided byremote virtual simulator 115 in FIG. 1, alternative embodiments may usevirtual rendering device 160 instead to recreate virtual render 129 fromvirtual environment data 120.

Real environment 130 may comprise a real location to be captured bylocal video capture system 135, such as a sports stadium, playing field,indoor court, outdoor field, or another locale. Objects to be tracked inreal environment 130 such as people, boundaries, balls, and otherobjects may be tracked with some form of location tracking system suchas a location tracking system using radio frequency identification(RFID) tags. Objects to be tracked may be moving or static. Thislocation tracking system may be integrated as part of local videocapture system 135.

As with virtual environment data 120, virtual rendering device 160 mayrequest a one-time or continuous data feed for real environment 130,which is provided by real environment data 140. Real object 141 mayinclude location and other data concerning an object within realenvironment 130. Real coordinates 142 may track the position of realobject 141 within the three-dimensional space of real environment 130.Metadata 143 may further contain other data regarding real object 141,such as translation data 141 regarding rotation and orientation of theobject, motion data 145 including direction vectors or trajectories, andpriority data 146 providing visibility rules for interferences withother objects. Video capture 149 may comprise a real-time video feed ofreal environment 130 provided by video cameras of local video capturesystem 135. Although in FIG. 1, virtual rendering device 160 accessesreal environment data 140 directly through local video capture system135, alternative embodiments may access real environment data 140through a network source accessible from network 150.

Virtual rendering device 160 may comprise a general purpose processingsystem executing custom software, or a special purpose hardware unit,and includes a memory (not shown) for storage of instructions and datafor the operation of virtual rendering device 160. Once processor 161establishes data feeds for virtual environment data 120 and realenvironment data 140, it may utilize compositing logical 164 andgraphical overlay 162 to generate composite render 165. Graphic overlay162 may include, for example, text windows, marquees, diagrams, arrows,telestration markings, and other information of interest. Since remotevirtual simulator 115 provides virtual coordinates 122, overlaycoordinates 163 can follow the path of virtual object 121 as thesimulation progresses. Similarly, real coordinates 142 can also be usedto reposition overlay coordinates 163 in real-time.

Moreover, by utilizing the data provided in virtual environment data 120and real environment data 140, processor 161 can automatically ensurethat overlay coordinates 163 are selected to avoid unwantedinterferences with real or virtual objects from real environment 130 andvirtual environment 110. Priority rules can also specify whether virtualor real objects should display first, and in which order. In thismanner, graphic overlay 162 can appear to follow a virtual or realobject while concurrently avoiding collisions or interferences withother displayed objects. Since processor 161 handles the calculation ofoverlay coordinates 163, mistakes resulting from manual repositioningcan be avoided, and more accurate tracking can be provided, even inreal-time. Since graphic overlay 162 may use data from dynamic datasources such as databases, processor 161 may also generate the imageportion of graphic overlay 162 in real-time from internal or externaldata sources.

Once graphic overlay 162, including overlay coordinates 163, isgenerated, processor 161 may apply compositing logic 164 to combinevirtual render 129, video capture 149, and graphic overlay 162 ascomposite render 165. Compositing logic 164 may contain rules and otheralgorithms for deciding how to combine real, virtual, and overlayelements within a single frame. Composite render 165 can then be sentvia live broadcast link 155 to be shown on display 170. Live broadcastlink 155 may comprise, for example, a satellite uplink to a televisionstudio, from where it is disseminated to the general public. Display 170may then represent, for example, a television of a viewer watching alive sports program.

In this manner, broadcasts combining live footage with virtualenvironments may be presented to viewers, opening up new possibilitiesfor sports analysis, scenario enactments, strategy presentations, andother augmented reality applications. Additionally, with graphicoverlays that intelligently follow virtual or real objects whileautomatically avoiding colliding with or obscuring other onscreenobjects, viewers can be apprised of game statistics, player data andportraits, and other relevant and topical information in a dynamicmanner, at onscreen positions most likely to engage the interest ofviewers. Since there is no need for manual adjustment to optimallyposition the graphic overlays, human error and operator costs can bereduced or eliminated compared to traditional manual overlaypositioning.

FIG. 2 presents a diagram of real and virtual environments for use by asystem for rendering in accordance with location of virtual objects inreal-time, according to one embodiment of the present invention. Diagram200 of FIG. 2 includes virtual environment 210, real environment 230,virtual rendering device 260, and composite render 265. Virtualenvironment 210 includes virtual objects 221 a-221 b. Real environment230 includes real objects 241 a-241 b and background 231. With regardsto FIG. 2, it should be noted that virtual environment 210 correspondsto virtual environment 110 from FIG. 1, that real environment 230corresponds to real environment 130, that virtual rendering device 260corresponds to virtual rendering device 160, and that composite render265 corresponds to composite render 165.

Real environment 230 depicts a scene of a football field captured byvideo cameras. Background 231 shows the faces of spectators in thebackground, real object 241 a shows a quarterback with the uniformnumber 5, and real object 241 b shows a linebacker with the uniformnumber 55. The positions of real objects 241 a-241 b might be trackedthrough RFID tags attached to player uniforms, as previously discussed.

Virtual environment 210 corresponds to a virtual version of the footballfield captured by real environment 230. Virtual object 221 a correspondsto a virtual version of real object 241 a, and virtual object 221 bcorresponds to a virtual version of a real object not shown in realenvironment 230, but may comprise a wide receiver with the uniformnumber 85.

As previously discussed, virtual rendering device 260 can combine dataextracted from real environment 230 and virtual environment 210 togenerate composite render 265, to form a kind of augmented realitybroadcast mixing both real and virtual elements. As shown by compositerender 265, real object 241 a takes precedence over virtual object 221a. This may be a result of a global rule prioritizing real objects overvirtual objects, or by specific priority rules associated withparticular objects. Although a graphic overlay is not shown in compositerender 265, virtual rendering device 260 may also insert a graphicoverlay to provide additional information to viewers, discussed inconjunction with FIG. 3 below.

FIG. 3 presents a series of exemplary interfaces on a display using asystem for rendering in accordance with location of virtual objects inreal-time, according to one embodiment of the present invention. Diagram300 of FIG. 3 includes displays 370 a-370 c. Display 370 a includesbackground 331, real objects 341 a-341 c, and graphic overlay 362.Display 370 b includes virtual object 321 a-321 b, background 331, realobjects 341 a-341 c, and graphic overlay 362. Display 370 c includesvirtual objects 321 a-321 c, background 331, real objects 341 a-341 c,and graphic overlay 362. With regards to FIG. 3, it should be noted thatdisplays 370 a-370 c correspond to display 170 from FIG. 1.

Display 370 a shows an arrangement similar to real environment 230 ofFIG. 2, with the addition of graphic overlay 362 and the line ofscrimmage marked as real object 341 c. As shown by graphic overlay 362,a text window displaying data regarding real object 341 a is shown toviewers, including the name of the player or “John Doe,” the uniformnumber or #5 quarterback, the completion percentage or 70%, and thedistance of the player from the line of scrimmage as real object 341 c,or 7 yards. As shown in display 370 a, graphic overlay 362 is positionedadjacent to the right of real object 341

Moving down to display 370 b, which may represent a state at a latertime, virtual objects 321 a-321 b are introduced, distinguishable fromreal objects by crossed line shading. In an actual rendering, virtualobject 321 a might be rendered as a realistic three-dimensional model asused in videogames and computer generated (CG) animations. The arrowpointing from real object 341 a to virtual object 321 a may represent asimulated movement path for John Doe, and may also be shown onscreen toprovide a visual indicator for viewers.

Furthermore, graphic overlay 362 may track the position of virtualobject 321 a as it moves across display 370 b, using coordinatesprovided by both real and virtual environments comprising display 370 b.Moreover, as shown in display 370 b, graphic overlay 362 is positionedabove virtual object 321 a such that it does not obscure real objects341 a-341 b. This may be implemented by, for example, a global ruleprioritizing the display of real objects corresponding to players toprevent graphic overlays from obscuring the position of the players.Since background 331 is not a player, it might be given a low priority,thereby allowing graphic overlay 362 to overlap background 331.

Moving to a later time at display 370 c, the simulation further has thevirtual version of John Doe throwing a virtual football represented byvirtual object 321 c to Jack Deer represented by virtual object 321 b.Since the focus of the simulation has moved from John Doe throwing apass to Jack Deer receiving the pass, the focus of graphic overlay 362may migrate to virtual object 321 b representing Jack Deer. Thus, asshown in display 370 c, the text window now provides informationregarding virtual object 321 b, including the name of the player, “JackDeer,” the number of his uniform, number 85 wide receiver, and hisaverage reception distance, or 10.2 yards. Again, using priority rulesas discussed above, the position of graphic overlay 362 may be adjustedsuch that it does not interfere with other onscreen objects, providingrelevant information at the most convenient location without obscuringthe action onscreen.

Of course, graphic overlay 362 is not limited to tracking players ortheir virtual representations, and could also track, for example, thefootball, or virtual object 321 c, while in flight, relaying statisticssuch as speed, spin, and distance until the next down or from the lineof scrimmage as real object 341 c. Graphic overlay 362 might alsoprovide information for static objects such as lines and boundaries.Moreover, although only a single graphic overlay 362 has been utilizedso far, multiple graphic overlays could also be used to track multipleobjects within a display. In this manner, potential play strategies andother useful and interesting commentary can be provided even inreal-time, with the previously discussed automatic graphic overlaypositioning mechanism largely removing any risk of covering up importantreal game footage or important virtual objects.

FIG. 4 shows a flowchart describing the steps, according to oneembodiment of the present invention, by which a rendering device canrender in accordance with location of virtual objects in real-time.Certain details and features have been left out of flowchart 400 thatare apparent to a person of ordinary skill in the art. For example, astep may comprise one or more substeps or may involve specializedequipment or materials, as known in the art. While steps 410 through 440indicated in flowchart 400 are sufficient to describe one embodiment ofthe present invention, other embodiments of the invention may utilizesteps different from those shown in flowchart 400.

Referring to step 410 of flowchart 400 in FIG. 4 and environment 100 ofFIG. 1, step 410 of flowchart 400 comprises virtual rendering device 160receiving virtual coordinates 122 of virtual object 121 in virtualenvironment 110. As previously discussed, this may be accomplishedthrough access via network 150 as shown in FIG. 1, or by a directconnection to remote virtual simulator 115, or by integrating remotevirtual simulator 115 with virtual rendering device 160.

Referring to step 420 of flowchart 400 in FIG. 4 and environment 100 ofFIG. 1, step 420 of flowchart 400 comprises virtual rendering device 160determining overlay coordinates 163 in accordance with virtualcoordinates 122 of virtual object 121 obtained from step 410. Forexample, processor 161 may apply a rule to position graphic overlay 162directly adjacent to virtual object 121, with a relative directiondepending on the position of virtual object 121 when rendered onto adisplay. A less complicated rule such as placing directly adjacent belowmight be used as well. Either way, virtual coordinates 122 are used insome manner to derive overlay coordinates 163.

Referring to step 430 of flowchart 400 in FIG. 4 and environment 100 ofFIG. 1, step 430 of flowchart 400 comprises virtual rendering device 160tracking a movement of virtual object 121 in virtual environment 110 byreceiving one or more three-dimensional coordinates of virtual object121. In step 430, virtual rendering device 160 continues to read a datafeed of virtual environment data 120 where virtual coordinates 122 ofvirtual object 121 are updated according to a running simulation atremote virtual simulator 115. For example, examining diagram 300 of FIG.3, if virtual object 121 corresponds to virtual object 321 a depicting asimulated movement of real object 341 a, then between a time period fromdisplays 370 a to 370 b, step 430 may have processor 161 of virtualrendering device 160 receiving a series of updated coordinates forvirtual coordinates 122 reflecting the movement of virtual object 321 aas shown by the arrow in display 370 b. These updates might be providedat a rate equal to or higher than the framerate of live broadcast link155 to display 170, or interpolation might be used to estimate positionsbetween updates.

Referring to step 440 of flowchart 400 in FIG. 4 and environment 100 ofFIG. 1, step 440 of flowchart 400 comprises virtual rendering device 160modifying overlay coordinates 163 of graphic overlay 162 determined fromstep 420 in accordance with the one or more three-dimensionalcoordinates of virtual object 121 tracked from step 430. As previouslydiscussed, the positioning of overlay coordinates 163 relative tovirtual coordinates 122 might be defined by an advanced rule taking intoaccount the position of virtual coordinates 122 as projected ontodisplay 170, or a simpler rule always selecting an adjacent locationoffset from a single directional vector. In either case, therelationship between overlay coordinates 163 and virtual coordinates 122should be such that their correspondence is clearly visible whenassociated with a virtual object and a graphic overlay on display 170.

Furthermore, as previously discussed, virtual rendering device 160 mayapply various priority rules and other mechanisms for avoiding overlapand collision with other objects onscreen that may clutter display 170and present a less than ideal overview of virtual environment 10 andreal environment 130. As previously discussed, one example might be toprioritize the display of real players, so that graphic overlays neverobscure the position of the players on the field, which may be veryimportant for watching a game in progress. Since processor 161 is ableto accomplish this positioning automatically with the help ofcompositing logic 164 and data feeds provided by virtual environmentdata 120 and real environment data 140, image overlays can now bereadily rendered in accordance to the location of virtual objects inreal-time, providing new and exciting ways of presenting valuable andrelevant information to viewers in real-time at optimal positionsonscreen without the need for error prone manual placements andadjustments.

From the above description of the invention it is manifest that varioustechniques can be used for implementing the concepts of the presentinvention without departing from its scope. Moreover, while theinvention has been described with specific reference to certainembodiments, a person of ordinary skills in the art would recognize thatchanges can be made in form and detail without departing from the spiritand the scope of the invention. As such, the described embodiments areto be considered in all respects as illustrative and not restrictive. Itshould also be understood that the invention is not limited to theparticular embodiments described herein, but is capable of manyrearrangements, modifications, and substitutions without departing fromthe scope of the invention.

What is claimed is:
 1. A method for persistent association of a graphicoverlay with a first virtual object in a displayable environment, themethod comprising: receiving a first three-dimensional coordinate of thefirst virtual object in the displayable environment; determining athree-dimensional coordinate of the graphic overlay in accordance withthe first three-dimensional coordinate of the first virtual object,wherein the determining the three-dimensional coordinate of the graphicoverlay applies positional rules so that the graphic overlay does notobscure both the first virtual object and one or more other objects inthe displayable environment; tracking a movement of the first virtualobject in the displayable environment by receiving one or more secondthree-dimensional coordinates of the first virtual object from a remotevirtual simulator; modifying the three-dimensional coordinate of thegraphic overlay in accordance with the one or more secondthree-dimensional coordinates of the first virtual object, whereinmodifying the three-dimensional coordinate of the graphic overlayincludes repositioning the graphic overlay to avoid obscuring the firstvirtual object and the one or more other objects in the displayableenvironment; and rendering the graphic overlay and the displayableenvironment for displaying on a display.
 2. The method of claim 1further comprising: receiving a first three-dimensional coordinate of areal object in the displayable environment further-modifying thethree-dimensional coordinate of the graphic overlay in accordance withthe first three-dimensional coordinate of the real object.
 3. The methodof claim 2, wherein the real object is a static object, and the one ormore other_objects are virtual objects.
 4. The method of claim 1 furthercomprising: receiving a first three-dimensional coordinate of a secondvirtual object in the displayable environment further modifying thethree-dimensional coordinate of the graphic overlay in accordance withthe first three-dimensional coordinate of the second virtual object. 5.The method of claim 1 , wherein the determining the three-dimensionalcoordinate of the graphic overlay further uses metadata of the firstvirtual object.
 6. The method of claim 5, wherein the metadata includestranslation data.
 7. The method of claim 5, wherein the metadataincludes motion data.
 8. The method of claim 5, wherein the metadataincludes priority data.
 9. The method of claim 1, wherein the graphicoverlay comprises text window.
 10. The method of claim 1, wherein thegraphic overlay comprises a graphical arrow.
 11. A rendering device forrendering a displayable environment with persistent association of agraphic overlay with a first virtual object, the rendering devicecomprising: a processor configured to: receive a first three-dimensionalcoordinate of the first virtual object in the displayable environment;determine a three-dimensional coordinate of the graphic overlay inaccordance with the first three-dimensional coordinate of the firstvirtual object, wherein the three-dimensional coordinate of the graphicoverlay is determined using positional rules so that the graphic overlaydoes not obscure both the first virtual object and one or more otherobjects in the displayable environment; track a movement of the firstvirtual object in the displayable environment by receiving one or moresecond three-dimensional coordinates of the first virtual object from aremote-virtual simulator; modify the three-dimensional coordinate of thegraphic overlay in accordance with the one or more secondthree-dimensional coordinates of the first virtual object, whereinmodifying the three-dimensional coordinate of the graphic overlayincludes repositioning the graphic overlay to avoid obscuring the firstvirtual object and the one or more other objects in the displayableenvironment; and render the graphic overlay and displayable environmenton a display.
 12. The rendering device of claim 11, wherein theprocessor is further configured to: receive a first three-dimensionalcoordinate of a real object in the displayable environment furthermodifying the three-dimensional coordinate of the graphic overlay inaccordance with the first three-dimensional coordinate of the realobject.
 13. The rendering device of claim 12, wherein the real object isa static object, and the one or more other objects are virtual objects.14. The rendering device of claim 11, wherein the processor is furtherconfigured to: receive a first three-dimensional coordinate of a secondvirtual object in the displayable environment further modifying thethree-dimensional coordinate of the graphic overlay in accordance withthe first three-dimensional coordinate of the second virtual object. 15.The rendering device of claim 11, wherein the processor is furtherconfigured to determine the three-dimensional coordinate of the graphicoverlay by using metadata of the first virtual object.
 16. The renderingdevice of claim 15, wherein the metadata includes translation data. 17.The rendering device of claim 15, wherein the metadata includes motiondata.
 18. The rendering device of claim 15, wherein the metadataincludes priority data.
 19. The rendering device of claim 11, whereinthe graphic overlay comprises a text window.
 20. The rendering device ofclaim 11, wherein the graphic overlay comprises a graphical arrow.